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The MAILING DATE of this communication appears on the cover sheet with the correspondence address ~ 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )[X] Responsive to communication(s) filed on 23 May 2001 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [X] Claim(s) 1-23 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) __ is/are allowed. 

6) [X] Claim(s) 1-23 is/are rejected. 

7) Q Claim(s) , is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) d The specification is objected to by the Examiner. 

10)[X] The drawing(s) filed on 23 May 2001 is/are: a)[X] accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121(d). 
1 1 )D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)d All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . Claims 1 - 23 are presented for examination. 
Claims 1-23 are rejected. 

Claim Objections 

2. Claim 4 is objected to because of the following informalities: According to the 
limitation of claim 4, "the step of training the text search detector", claim 4 should be one 
dependent claim depending on independent claim 1. Claim 4 should be added "The method of 
claim 1" to show it is depending on independent claim 1. Appropriate correction is required. 

3. Claim 14 is objected to because of the following informalities: Claim 14 depending on 
claim 14 itself is not appropriate. According to the limitation of claim 14 "criteria for declaring 
an approximate correlation between two the records, A and B". Claim 14 should depend on 
claim 13. Appropriate correction is required. 

Claim Rejections - 35 USC $ 112 

The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

4. Claims 1-17 are rejected under 35 U.S.C. 112, second paragraph; there is 
insufficient antecedent basis for this limitation in the claim. 
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5. As to claim 1-17, there is insufficient antecedent basis for this limitation in the claim. 
For example, applicants recite the limitation "the event" of claim 1 [line 6 and line 9]; claim 15 
[line 1] is not previously cited. 

6. As to dependent claim 11, applicants recite the limitation "wherein the target within the 
image" in first line of claim 11. There is insufficient antecedent basis for this limitation in the 
claim. The "the target within the image", in first line of claim 1 1, is not previously cited. 

7. As to dependent claim 13 and 14, applicants recite the limitation "wherein criteria for 
declaring an approximate correlation between two the records, A and B" in line 1 - 2 of claim 
13. There is insufficient antecedent basis for this limitation in the claim. The "criteria", in first 
line of claim 13 and 14, is not previously cited. 

8. As to dependent claim 16, applicants recite the limitation "object" in first line of claims 
16 and 22. There is insufficient antecedent basis for this limitation in the claim. The "object", in 
first line of claim 16, is not previously cited. 

Claim Rejections - 35 USC S 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 
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9. Claims 1 - 6, 11, and 15-23 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Richard L. Delanoy (U.S. Patent No. 5,793,888, and referred to as Delanoy hereinafter). 

10. Regarding independent claim 1, Delanoy teaches, a method of correlating text and 
imagery [recognize patterns of spectral or textural features in imagery, Abstract, Line 1-2], 
steps of specifying a target concept [setting forth a user-desired image subject, Col. 3, Line 10]; 
providing textual material and imagery [various attributes, Col. 3, Line 11]; training 
[determining, Col. 3, Line 12] a text search detector to examine [processor means, Col. 3, Line 
12] the textual material for text regions [various attributes, Col. 3, Line 13] which relate to the 
target concept [user-desired, Col. 3, Line 10], .and creating a text target detection record 
[attributes, Col. 3, Line 14] in a database A [subset, Col. 3, Line 13] in the event of a match or 
other meaningful association [most discriminating, Col. 3, Line 14 - 15]; training a 
discriminating feature detector [functional template member, Col. 3, Line 15] to search for 
locations [providing a functional template, Col. 3, Line 16] within the imagery which relate to 
the target concept [for each discriminating attributes, Col. 3, Line 17], and creating a location 
target detection record in a database B [functional template, Col. 3, Line 16] in the event of a 
match or other meaningful association [based on scoring functions, Col. 3, Line 17]; and 
comparing [matching, Col. 3, Line 20] the records in both databases [subset, Col. 3, Line 13 and 
functional template, Col. 3, Line 16] to declare an approximate correlation [extract user desired 
images, Col. 3, Line 20 - 21, if any, indicative of a common target concept [employs the 
functional template in a matching routine, Col. 3, Line 19-20]. 
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1 1 . Regarding dependent claims 2 and 20, Delanoy teaches, the target concept is an event 
or object [image subject, Col. 3, Line 10]. 

12. Regarding dependent claim 3, Delanoy teaches, the discriminating features [data 
source, Col. 5, Line 16] within the imagery include infrared, multispectral or spatial features 
[multispectral, Col. 5, Line 16]. 

13. Regarding dependent claim 4, Delanoy teaches, the steps: a) defining a search phrase 
[attributes of the subject, Col. 5, Line 2]; b) testing the phrase against a validation set [supervised 
learning session, Col. 5, Line 4], and c) repeating a) and b) until all relevant targets in the 
validation set are detected [continuously provide feedback to each other, Col. 5, Line 5-6]. 

14. Regarding dependent claim 5, Delanoy teaches, the examination of the textual material 
[determines various attributes, Col. 3, Line 13] includes searching the text regions [user desired 
image subject, Col. 4, Line 1 1 - 12] for geographic location text associated with the target 
concept [an object, geographical area, geometrical shape, Col. 4, Line 12-13]. 

15. Regarding dependent claim 6, Delanoy teaches, generating a concept identifier code 
[agent, Col. 3, Line 19] in both the text and image target detection records [attributes, Col. 3, 
Line 1 1 and functional template, Col. 3, Line 15] using a lookup table in the event of a match or 
other meaningful association [matching routing, Col. 3, Line 21]. 
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16. Regarding dependent claim 11, Delanoy teaches, target within the image [spatial 
properties, Col. 8, Line 40] is in the form of a pixel index [each pixel an index, Col. 8, Line 39]. 

17. Regarding dependent claims 15 and 21, Delanoy teaches, the event may be 
characterized as an emergency, tragedy, disaster or crisis [the user inputs information about the 
data and the nature of the object or region to be sought, Col. 5, Line 7 - 8]. 

18. Regarding dependent claims 16 and 22, Delanoy teaches, the object [image, Col. 4, 
Line 66] involves an environmental asset, structure, or mode of transportation [containing some 
subject, object or region, or subimage, Col 4, Line 65 - 67], 

19. Regarding dependent claim 17, Delanoy teaches, steps associated with examining 
[supporting program revises, Col. 5, Line 51] the textual material or searching for locations 
within the imagery [model of user intensions, Col. 5, Line 51-52] are earned out in a batch 
mode or as part of a recursive flow [the process repeats until the user is satisfied with the agent's 
performance, Col. 5, Line 52 - 54]. 

20. Regarding independent claim 18, Delanoy teaches, a text and imagery spatial correlator 
[an interest image is a spatial map of evidence, Col 6, Line 54], a document text parsing and 
interpretation engine [detection algorithm, Col. 7, Line 3] which uses a context-based [data 
fusion, Col. 7, Line 6] search to generate topical information [provide a means of representing 
information for human interpretation, Col. 7, Line 6 - 7]; an imagery engine operative 
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[functional template correlation, Col. 7, Line 11, 30] to associate [consists, Col. 7, Line 31] the 
components of an image [subimage, Col. 7, Line 32] with known spatial features and generate 
location information [input image values, Col. 7, Line 35]; and a matching subsystem 
[autocorrelation, Col. 7, Line 17] operative to associate [matching, Col. 7, Line 19] the topical 
information [kernel K, Col. 7, Line 19] with the location information [pixel location, Col. 7, Line 
20] and present a result to a user [output, Col. 7, Line 18]. 

21. Regarding dependent claim 19, Delanoy teaches, text parsing and interpretation engine 
[a set of simple feature detectors, Col. 7, Line 4] includes a user-trainable agent [detection 
algorithm, Col. 7, Line 3] to define the context of interest in a current search [provide a means of 
representing information for human interpretation, Col. 7, Line 6-7]. 

22. Regarding dependent claim 23, Delanoy teaches, the matching subsystem [supporting 
program, Col. 5, Line 50] is operative to associate the topical information [user's intention, Col. 
5, Line 50] with the location information in a batch mode or as part of a recursive flow [the 
process repeats until the user is satisfied with the agent's performance, Col. 5, Line 52 - 54]. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

23. Claims 7 - 10, 12, and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Delanoy, and in view of Margaret E. Elliott (U.S. PGPUB 2002/0156779, and referred to as 
Elliott hereinafter). 

(Delanoy as set forth above generally discloses the basic inventions.) 

24. Regarding dependent claim 7, Delanoy teaches the searching of the test regions 
[various attributes, Col. 3, Line 13]. 

Delanoy does not teach accomplished by reference to a Gazetteer of place names and 
their corresponding lat-long locations. 

Elliott teaches accomplished by reference to a Gazetteer of place names [gazetteer, 0031, 
Line 5] and their corresponding lat-long locations [longitude/latitude coordinates, 0022, Line 5]. 
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It would have been obvious to a person of ordinary skill in the art at the time of 
applicant's invention to modify the teaching of Delanoy to include "accomplished by reference 
to a Gazetteer of place names and their corresponding lat-long locations" for the purpose of 
indexing information against a database of spatial language which is used in combination with a 
modified search engine that conducts searches using spatially relevant criteria and spatial 
analysis algorithms Alpha-numeric values from a mathematical system are used for identifying 
spatial locations, and can be arbitrary, geocentric, virtual, and galactic. 

25. Regarding dependent claim 8, Delanoy teaches the text target detection record 
[attributes, Col. 3, Line 14]. 

Delanoy does not teach a text document ID number, an index to locate a paragraph or 
passage within the document, the target concept identifier code (CIC), and the latitude-longitude 
(LL) value. 

Elliott teaches a text document ID number [Identifier information, 0021, Line 3], an 
index to locate a paragraph or passage within the document [attribute information, 0021, Line 7], 
the target concept identifier code (CIC) [identifier code, 0021, Line 8], and the latitude-longitude 
(LL) value [longitude/latitude coordinates, 0022, Line 5]. 

It would have been obvious to a person of ordinary skill in the art at the time of 
applicant's invention to modify the teaching of Delanoy to include "a text document ID number, 
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an index to locate a paragraph or passage within the document, the target concept identifier code 
(CIC), and the latitude-longitude (LL) value" for the purpose of indexing information against a 
database of spatial language which is used in combination with a modified search engine that 
conducts searches using spatially relevant criteria and spatial analysis algorithms Alpha-numeric 
values from a mathematical system are used for identifying spatial locations, and can be 
arbitrary, geocentric, virtual, and galactic. 

26. Regarding dependent claim 9, Delanoy teaches the search for locations within the 
imagery [providing a functional template, Col. 3, Line 16]. 

Delanoy does not teach extracting a lat-long location. 

Elliott teaches extracting a lat-long location [longitude/latitude coordinates, 0022, Line 

5]. 

It would have been obvious to a person of ordinary skill in the art at the time of 
applicant's invention to modify the teaching of Delanoy to include "extracting a lat-long 
location" for the purpose of indexing information against a database of spatial language which is 
used in combination with a modified search engine that conducts searches using spatially 
relevant criteria and spatial analysis algorithms Alpha-numeric values from a mathematical 
system are used for identifying spatial locations, and can be arbitrary, geocentric, virtual, and 
galactic. 
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27. Regarding dependent claim 10, Delanoy teaches the location target detection record 
[functional template, Col. 3, Line 16]. 

Delanoy does not teach an image ID number, an index to locate the target within the 
image, the target concept identifier code (CIC), and the latitude-longitude (LL) value. 

Elliott teaches an image ID number [Identifier information, 0021, Line 3], an index to 
locate the target within the image [attribute information, 0021, Line 7], the target concept 
identifier code (CIC) [identifier code, 0021, Line 8], and the latitude-longitude (LL) value 
[longitude/latitude coordinates, 0022, Line 5]. 

It would have been obvious to a person of ordinary skill in the art at the time of 
applicant's invention to modify the teaching of Delanoy to include "an image ID number, an 
index to locate the target within the image, the target concept identifier code (CIC), and the 
latitude-longitude (LL) value" for the purpose of indexing information against a database of 
spatial language which is used in combination with a modified search engine that conducts 
searches using spatially relevant criteria and spatial analysis algorithms Alpha-numeric values 
from a mathematical system are used for identifying spatial locations, and can be arbitrary, 
geocentric, virtual, and galactic. 

28. Regarding dependent claim 12, Delanoy teaches the searching of the test regions 
[determining, Col. 3, Line 12]; and the text target detection record [attributes, Col. 3, Line 14]. 
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Delanoy does not teach reference to a Gazetteer of place names and their corresponding 
lat-long locations; and the text target detection record contains: a text document ID number, an 
index to locate a paragraph or passage within the document, the target concept identifier code 
(CIC), and the latitude-longitude (LL) value. 

Elliott teaches reference to a Gazetteer of place names [gazetteer, 0031, Line 5] and their 
corresponding lat-long locations [longitude/latitude coordinates, 0022, Line 5]; and the text 
target detection record contains: a text document ID number [Identifier information, 0021, Line 
3], an index to locate a paragraph or passage within the document [attribute information, 0021, 
Line 7], the target concept identifier code (CIC) [identifier code, 0021, Line 8], and the 
latitude-longitude (LL) value [longitude/latitude coordinates, 0022, Line 5]. 

It would have been obvious to a person of ordinary skill in the art at the time of 
applicant's invention to modify the teaching of Delanoy to include "reference to a Gazetteer of 
place names and their corresponding lat-long locations; and a text document ID number, an 
index to locate a paragraph or passage within the document, the target concept identifier code 
(CIC), and the latitude-longitude (LL) value" for the purpose of indexing information against a 
database of spatial language which is used in combination with a modified search engine that 
conducts searches using spatially relevant criteria and spatial analysis algorithms Alpha-numeric 
values from a mathematical system are used for identifying spatial locations, and can be 
arbitrary, geocentric, virtual, and galactic. 
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29. Regarding dependent claim 13, Delanoy teaches a method of correlating text and 
imagery [recognize patterns of spectral or textural features in imagery, Abstract, Line 1-2] 

Delanoy does not teach IF CIC in record A = CIC in record B, THEN record A and 
record B are associated. 

Elliott teaches a pattern of characters with wildcards, can represent different string 
combinations which have the same meaning. This allo B ws the spider to check if a particular string 
matches a given regular expression or if a given regular expression matches a particular string 
[0068, Line 11-15]. 

It would have been obvious to a person of ordinary skill in the art at the time of 
applicant's invention to modify the teaching of Delanoy to include "IF CIC in record A = CIC in 
record B, THEN record A and record B are associated" for the purpose of indexing information 
against a database of spatial language which is used in combination with a modified search 
engine that conducts searches using spatially relevant criteria and spatial analysis algorithms 
Alpha-numeric values from a mathematical system are used for identifying spatial locations, and 
can be arbitrary, geocentric, virtual, and galactic. 
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30. Regarding dependent claim 14, Delanoy teaches a method of correlating text and 
imagery [recognize patterns of spectral or textural features in imagery, Abstract, Line 1-2] 

Delanoy does not teach IF (LL) in record A is within S of (LL) in record B, THEN record 
A and record B are associated. Where S is a user-selectable spatial distance. 

Elliott teaches IF (LL) in record A is within S of (LL) in record B, THEN record A and 
record B are associated. Where S is a user-selectable spatial distance. [A spatial search engine 
searches the spatial index to that index of words or the spatial columns of data in that index of 
words to find matches in a radius distance from a geographic coordinate . Col. 2, 0015, Line 2 - 
5] 

It would have been obvious to a person of ordinary skill in the art at the time of 
applicant's invention to modify the teaching of Delanoy to include "IF (LL) in record A is within 
S of (LL) in record B, THEN record A and record B are associated. Where S is a user-selectable 
spatial distance" for the purpose of indexing information against a database of spatial language 
which is used in combination with a modified search engine that conducts searches using 
spatially relevant criteria and spatial analysis algorithms Alpha-numeric values from a 
mathematical system are used for identifying spatial locations, and can be arbitrary, geocentric, 
virtual, and galactic. 
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Conclusion 



3 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Shunji Matsumoto et al. (U.S. Patent No. 5,638,460) disclose inputting image using 
fuzzy logic, receiving user input string, selecting keywords, calculating correlation factors. 
Richard L. Delanoy et al. (U.S. Patent No. 5,222,155) disclose image point value, template point 
value, scoring function, fuzzy template shape matching. Mario Marco Degasperi (U.S. Patent 
No. 4,075,604) discloses real time image recognition, comparison with sample images, 

32. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sunray Chang whose telephone number is 703-305-8744. The 
examiner can normally be reached on M-F 7:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Anthony Knight can be reached on (703)308-3179 . The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-746-3506. 

Sunray Chang 

Patent Examiner _ J /) , * 
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